Recording and reproducing system for image data with recording position information and a recording and reproducing method therefor

ABSTRACT

An image data recording and reproducing system comprises an image data input unit, an ID generator generating an ID inserted to image data from the image data input unit, a plurality of image data recorders for recording the image data, and an output unit for outputting the image data recorded into the plurality of image data recorders, wherein each of the plurality of image data recorders has an ID table and an image data storage area, and an ID of the ID table is matched with image data of the image data storage area on the bases of recording position information of the image data.

RELATED APPLICATIONS

This is a continuation application of U.S. patent application Ser. No.10/777,096 filed Feb. 13, 2004, now allowed, the entire disclosure ofwhich is hereby incorporated by reference.

FIELD OF THE INVENTION

The present invention relates to a recording and reproducing system forimage data and a recording and reproducing method therefor. Morespecifically, the present invention relates to a recording andreproducing system for image data and a recording and reproducing methodtherefor capable of transmitting, recording and browsing image data onthe Internet or a local area network.

BACKGROUND OF THE INVENTION

In the image system field (hereinafter, called an image system)including a video surveillance system and an image delivery system, theneed for an image transmission unit transmitting image data on an IP(Internet Protocol) network such as the Internet or LAN (Local AreaNetwork) as a transmission line is increasing rapidly. In such imagesystem, a combination of a camera and an image transmission unit or anetwork camera (referring to a camera integrated with an imagetransmission unit) is used and a recorder (hereinafter, called a videoserver) directly recording image data therefrom via a network isbecoming mainstream. The above-described image transmission unit is aunit which, in order to transmit an image picked up by a camera to anetwork, compresses the image by MPEG2 (Moving Picture Expert GroupPhase 2) or MPEG4 to convert it to image stream data (hereinafter,called image data).

A prior art camera transmits image data in an analog form of the NTSCsystem as a TV standard system using a coaxial cable. The cameratransmits the image data as an analog signal so that deterioration dueto attenuation or noise occurs during transmission to degrade imagequality. In addition, a coaxial cable must be laid for each camera. Thecable laying cost is tremendous with longer distance. A remotesurveillance system in the actual use is significantly limited.

For image data from the above-described network camera or imagetransmission unit, an IP network such as the Internet or LAN has alreadybeen widely laid in a building or an area and no laying work laying anew IP network is necessary so as to build an image system very easily.As a result, connection to the Internet allows local image data to beviewed very easily in Japan and overseas. When laying a new IP network,wireless LAN is used to just place a relay station. No work providing atransmission line to a camera is necessary to significantly reduce thelaying cost.

General personal computer peripheral equipment can be used as-is asequipment for relay transmission. It is much inexpensive than an analogcamera. Because of an IP network system, system design in conjunctionwith various kinds of network components (such as a sensor, personalcomputer and server) is simplified. For example, in an entering/leavingmanagement system in a building using an IC card, face photograph imagedata of an entering/leaving person is picked up by a network camera, andthen, the frame image data of the person in the face photograph isstored together with an entering/leaving management database to build ahigh level of security network system in conjunction with theentering/leaving information and the image data.

The image system using an IP network has many advantages as describedabove. Since the technique and parts as a base are personal computerperipheral equipment, the image system has reliability lower than thatof the analog camera system. Unlike the NTSC system, the IP network as abase has an asynchronous communication system assuming data loss, forexample, packet discard when transmitting image data in a packet. Acomputer virus or sudden network load occurs to temporally interruptimage data. At worst, the entire image system can be down.

The personal computer peripheral equipment has equipment level moreunstable than that of the analog camera system. Along with higherfunction and complicating of video equipment such as a camera or arecorder, a high-function and complicated OS (Operating System) such asWindows (Microsoft's trademark) or Linux is employed, and freezing ofthe image system is found here and there. When requiring a high degreeof reliability in the image system, the network or equipment must besubject to duplicating or clustering like the server of a data center.

In the “duplicating” herein, another server performing quite the samefunction as a generally mainly operated server is prepared for backup,and the main server, which has failed, is switched to the backup serverfor operating the system continuously after failure. The backup servermust always be subject to mirroring of the main server. The mirroringherein means that the backup server into which the same data as that ofthe main server is written performs the same operation as that of themain server.

The “clustering” means that in a server configuration system forbalancing load concentrated on one server, a plurality of node servers(servers performing actual processing) and a load balancer are used toconfigure one virtual server. That is, a plurality of node serversconsist of Cluster. In the clustering, a failed server is disconnectedat failure, and then, the entire another plurality of servers shares theprocessing of the failed server to operate the system continuously afterfailure. To apply the clustering, individual node servers must have anequal function and must be subject to mirroring mutually. In addition,each job is independent (or each session is independent) like Webservers to limit the use of service in which no state management isnecessary between a client and a server.

In Web servers in which the duplicating technique and the clusteringtechnique are standardly used, each job is basically independent inoperation and no state management need be performed between a specificclient and server. In the event that fail over occurs, the client cansmoothly continue browsing without being affected thereby. The fail overherein means that two servers are operated at the same time and one ofthem is operated as a main server so that when the main server cannot beoperated, the second server-quickly takes over the processing as backup.

As a conventionally known multimedia recorder, for example, JapanesePatent Laid-Open Application No. Hei 11-250578 describes a plurality ofrecording devices and a backup device. The technique shows a techniqueenabling backup creation without deteriorating fast responsiveness atrecording required for event recording.

SUMMARY OF THE INVENTION

In the communication form between a client and a delivering side serverof the above-described network image delivery system, a system managingthe client on the delivering side is standard. In such system, when theservers must be switched due to some reason such as failure or overload,information managing the reproduction state is lost. Consequently, thestate before disconnection cannot be continued and the information canbe only obtained from the initial state again. The image dataimmediately before failure cannot be continuously browsed.

In the system using the duplicating technique or the clusteringtechnique, when database-related link information, for example,recording position information of recorded image data, is mismatchedbetween a plurality of servers, the recording positions of the sameimage data of the video servers having the same recording capacity areusually quite different depending on the used state. When the videoservers are switched due to an accident, the database-related linkinformation indicates quite non-related image data. When thedatabase-related link information is mismatched between the plurality ofvideo servers and the video servers are switched due to failure, the jobcannot be taken over smoothly between a specific client and server. Tobuild a highly reliable system, a set of databases must be unit betweenthe plurality of video servers for duplicating or clustering. The systemis so large that there is a significant problem in cost, operation, andsystem extensibility.

One object of the present invention is to provide a recording andreproducing system for image data and a recording and reproducing methodtherefor capable of equivalently accessing a plurality of video servers.

Another object of the present invention is to provide a recording andreproducing system for image data and a recording and reproducing methodtherefor capable of correctly using recording database-related linkinformation when distributively recording image data into a plurality ofvideo servers for duplicating or clustering.

Another object of the present invention is to provide a recording andreproducing system for image data and a recording and reproducing methodtherefor capable of minimizing the system resources of a video serverwhen recording link information into a database.

Another object of the present invention is to provide a recording andreproducing system for image data and a recording and reproducing methodtherefor capable of using recording database-related link informationwithout being affected by time variation.

Further object of the present invention is to provide a recording andreproducing system for image data and a recording and reproducing methodtherefor capable of integrally generating image data and linkinformation.

To achieve the above object, an image data recording and reproducingsystem of the present invention comprises an image data input unit, anID generator for generating an ID generated to image data from the imagedata input unit, a plurality of image data recorders for recording theimage data, and an output unit for outputting the image data recordedinto the plurality of image data recorders, wherein each of theplurality of image data recorders has an ID table and an image datastorage area, and an ID of the ID table is matched with image data ofthe image data storage area on the bases of recording positioninformation of the image data.

An image data recording and reproducing system of the present inventionfurther comprises a load balancer and an output terminal deviceconnected to the output unit, wherein the load balancer manages the loadfactor of each of the plurality of image data recorders and image datarequirement from said output terminal unit is transferred to any one ofthe predetermined plurality of image data recorders on the bases of theload factor of each of the plurality of image data recorders.

A recording and reproducing method for image data of the presentinvention comprises the steps of: generating a plurality of image data;and adding an identifyable ID to each of the plurality of image data;wherein when recording the image data with the ID added into a pluralityof image data recorders, each of the plurality of image data recordershas an ID table and an image data storage area, and an ID of the IDtable and image data of the image data storage area are storedcorresponding to recording position information of the image data toreproduce predetermined image data from image data recorded into theplurality of image data recorders based on the ID to command for a videoreplay.

The above and other objects, features and advantages of the presentinvention will be apparent from the following detailed description ofillustrative embodiments which is to be read in connection with theaccompanying drawings.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a diagram showing the schematic configuration of an embodimentof the present invention;

FIG. 2 is a diagram showing the specific configuration of an embodimentof the present invention shown in FIG. 1;

FIG. 3 is a diagram of assistance in explaining another embodiment ofthe present invention;

FIG. 4 is a diagram of assistance in explaining the schematicconfiguration of another embodiment of the present invention;

FIG. 5 is a diagram of assistance in explaining the schematicconfiguration of another embodiment of the present invention;

FIG. 6 is a diagram of assistance in explaining the schematicconfiguration of further embodiment of the present invention;

FIG. 7 is a diagram of assistance in explaining a table of data recordedinto an image data recorder of an embodiment of the present invention;and

FIG. 8 is a diagram showing a display screen of a client terminal usedfor the present invention.

DETAILED DESCRIPTION OF THE ILLUSTRATED EMBODIMENTS

FIG. 1 is an image recording and reproducing system diagram ofassistance in explaining an embodiment of the present invention. In FIG.1, the numeral 100 denotes an IP network (hereinafter, simply called anetwork, which may be a transmission line) performing communication bythe TCP (Transmission Control Protocol)/IP protocol such as wired LAN,wireless LAN or the Internet, the numeral 200 denotes an imagetransmission unit, and the numeral 300 denotes an image pick-up unitpicking up an object, e.g., a television camera. The numeral 400 denotesa sensor, e.g., an infrared sensor detecting the presence or absence ofan intruder in an intruder surveillance system or a water level sensordetecting the water level of a dam. The image transmission unit 200 is aunit converting the outputs of the camera 300 and the sensor 400 to IPdata to output them to the network 100. That is, it compresses the imagedata output of the camera 300 to a form such as JPEG, MPEG2 or MPEG4 toencode it for converting it to IP data, and superimposes sensorinformation from the sensor 400 on the IP image data for outputting itto the network 100.

The numeral 500 denotes a network camera. The network camera 500integrates the camera 300 with the image transmission unit 200. Imagedata from the network camera 500 is outputted as IP data directly to thenetwork 100. The numeral 600 denotes a video server and is a unitrecording image data transmitted from the image transmission unit 200 orthe network camera 500. The numeral 700 denotes a data recorder(database unit) for registering sensor information and link information(ID) specifying image data related to the sensor information when thesensor information is superimposed on the image data. Typically, thesurveillance system is given picking up dates of the image data by atimer incorporated into (or externally provided of) the camera 300 orthe network camera 500. These times are stacked onto the data recorder700 together with an ID.

The numeral 800 denotes a client terminal and is realized by a personalcomputer (PC) or an exclusive hardware configuration. The clientterminal 800 is used for reproducing live image data transmitted fromthe image transmission unit 200 or the network camera 500, reproducingimage data recorded into the video server 600, or reproducing datarecorded into the data recorder 700. The client terminal 800 has afunction for controlling, by a conventionally known method, theoperation of pan, tilt and zoom (A rotating mechanism such as a PT(Pan/Tilt) unit is not shown.) of the camera 300 or the network camera500 and the operation of the sensor 400, as necessary.

In FIG. 1, one image transmission unit 200, one camera 300, one sensor400 or one network camera 500 are representatively shown. Typically,several to several tens of image transmission units 200, cameras 300,sensors 400 or network cameras 500 are connected to the network 100 toconfigure the image recording and reproducing system. Also, typically,the client terminals 800 are installed in a plurality of places.

FIG. 2 is a block diagram of assistance in explaining the specificconfiguration of an embodiment of the present invention. In FIG. 2, thevideo server 600 consists of a plurality of image data recorders 600-1,600-2, . . . 600-n subject to clustering. The numeral 610 denotes an ID(Identification) (also called link information) table, and the numeral620 denotes an image data storage area. The ID table 610 and the imagedata storage area 620 are schematically shown and mean a storage arearecorded onto a recording medium such as a hard disk (HD), an opticaldisc or a DVD (Digital Versatile Disc) in the image data recorder 600-1.The ID table 610 may be stored in semiconductor memory when high-speedretrieval is required. The numeral 630 denotes a load balancer and thedetail will be described later. The numeral 700-1 denotes a datarecorder which is not necessarily required and is provided when needingto stack time information or sensor information, as described later. Thedata recorder 700-1 can be also used serving as one of the image datarecorders 600-n and need not be physically discriminated. The clientterminal 800 has an input unit 810, a decoder 820 and a display 830. Thenumeral 900 denotes an ID generator and the detail will be describedlater. The same things as FIG. 1 are designated by similar numerals. InFIG. 2, the network 100 is omitted and the transmission line between theimage transmission unit 200 and the video server 600 and thetransmission line between the client terminal 800 and the video server600 correspond to the network 100. The details of an ID will bedescribed later.

The system operation shown in FIG. 2 will be described. First, recordingof image data will be described. Image data outputted from the imagetransmission unit 200 is inputted to the ID generator 900. Variousmodifications of the installing place of the ID generator 900 can bemade. For convenience, the case that it is located between the imagetransmission unit 200 and the transmission line 100 will be describedhere. The ID generator 900 inserts an image data reference unit (or animage data reproduction unit), e.g., an ID (or ID information) for eachTV frame, into the image data outputted from the image transmission unit200, and outputs it to the network 100.

The image data to which the ID is inserted (or attached) is transmittedand recorded into all of the plurality of recorders 600-1, 600-2, . . .600-n. For example, in the image data recorder 600-1 as a node server,the image data is recorded into the image data storage area 620 on thehard disk in the server, and recording position information on thedecided area is matched with the inserted ID for recording into the IDtable 610.

The relation between the ID table 610 and the image data storage area620 will be described in detail using FIG. 7. FIG. 7( a) shows the IDtable 610 and FIG. 7( b) shows the recorded contents of the image datastorage area 620. When image data is inputted to the image data recorder600-1, it is stored in an area 621-1 on the image data storage area 620together with header address (position information) A1, as shown in FIG.7( b). An ID inserted to the image data and the position information A1of the area 621-1 are stored in a record 611-1 (indicated by a record611 for the generic name for the records) on the ID table 610, as shownin FIG. 7( a). That is, the ID (e.g., indicated by #101) and theposition information A1 are stored in the record 611-1.

When new image data is inputted, it is stored in an area 621-2 on theimage data storage area 620 together with header address (positioninformation) A2. For an ID inserted to the image data and the positioninformation A2 of the area 621-2, the ID (indicated by #102) and theposition information A2 are stored in a record 611-2 on the ID table610. Thereafter, each time new image data is inputted, it issequentially recorded into the image data storage area 620 and positioninformation Am (m: 1, 2, . . . ) of the image data storage area 620 andIDs (#M) (M=101, 102, . . . ) are sequentially stored in the record 611.

Each time image data is transmitted, the above processing is repeated.When there is no space in the image data storage area 620, old data aresequentially overwritten by the latest data. That is, the recordedcontents are updated. New data is added to the ID table 610 according tothis. When there is a record whose referred side is updated byoverwriting of the image data storage area 620, it is erased. The imagedata recorder 600-1 is described above in detail. Image data from the IDgenerator 900 and ID information inserted to each image data areinputted to the image data recorders 600-2, . . . 600-n as in the imagedata recorder 600-1 and are stored in the ID table and the image datastorage area owned by each of the image data recorders by the samemethod. The detailed description is omitted.

In the case of image data to which its picking up date is attached astime information, the time information is recorded into the datarecorder 700-1 together with an ID, as in FIG. 7. That is, the timeinformation is recorded into a location corresponding to the image datastorage area. The ID, time information and recording positioninformation of the time information are the same as those shown in FIG.7. The image data recorders 600-1, 600-2, . . . 600-n are large capacitystorage devices having a recording capacity enough for stackingpicked-up image data for about three days to one month.

Reproduction of image data will be described. When command for a videoreplay is given from the user to the client terminal 800, the input unit810 generates command for a video replay based on command for imagerequirement to transmit it to the network video server 600 subject toclustering. At this time, an ID specifying the recording location of therecorded image data is added to the command for image requirement. Theimage requirement input (query input) and the display screen of theclient terminal 800 will be described using FIG. 8.

The client terminal 800 has an input unit 810 such as a keyboard, adecoder 820 decoding image data, and a display 830, and an embodiment ofits retrieval and result display screen is shown in FIG. 8. The numeral850 in FIG. 8 is a retrieval and result display screen. The retrievaland result display screen 850 has a retrieval condition specificationscreen 851, a retrieval result list display screen 852, and areproduction screen 853.

Sensor 1 is specified as a retrieval item from the client terminal 800by the input unit 810 such as a keyboard on the retrieval conditionspecification screen 851 for command for a video replay. In this case,the sensor 1 corresponds to No. 1 camera of the image picking upcameras. In addition, retrieval start time: Jan. 1, 2002, 00:00 andretrieval end time: Mar. 31, 2002, 23:59 are inputted, downward order isinputted for time order, and 25 images are inputted for a number ofdisplayed retrieval result to click the retrieval button.

When the retrieval condition is inputted from the client terminal 800,the ID corresponding to the retrieval start time and the retrieval endtime of the retrieval condition is read from the data recorder 700-1,and the scaled-down images (thumbnail images) of image datacorresponding to the ID are read from the video server 600 via the loadbalancer 630 so as to be displayed on the retrieval result list displayscreen 852 as scaled-down images 854-1, 854-2, . . . 854-25 (Theretrieval result of 25 images is displayed). The scaled-down image(thumbnail image) 854 of the image data has pixels obtained by reducingthe number of pixels of one frame image and is an image for displaying alarge number of images in the retrieval result list display screen 852.In the scaled-down image 854, all the scaled-down images of a series ofpicked-up images are not displayed. A conventionally well-knownscaled-down image of a frame with changed images is selected anddisplayed as representative image data. The image picking-up date isalso displayed on the retrieval result list display screen 852 accordingto the need of the user.

The operation of the load balancer 630 will be described. When readingimage data from the image data recorders 600-1, 600-2, . . . 600-naccording to the inserted ID, the load balancer 630 refers to the loadstate of the video server 600 as the node server in the Cluster totransfer command for a video replay from the client 800 to the serverhaving the lowest load. The load balancer 630 switches the connectedsides according to the load of the image data recorders 600-1, 600-2, .. . 600-n to the command for a video replay from a plurality of clientterminals 800 connected to the network 100 and transfers the command fora video replay. For example, the load balancer 630 has in its inside atable of the image data recorders as shown in Table.1.

TABLE 1 Load factor (a Image data number of command for recorder No. avideo replay) 600-1 40% 600-2 100%  . . . . . . 600-n 60%

Table 1 shows a load factor for each of the image data recorders at apredetermined point. The load factor herein expresses a number ofcommand for a video replay accessed from the client terminal to each ofthe image data recorders as %. When command for a video replay is givenfrom the client terminal 800, the load balancer 630 refers to the table1 and transfers the command for a video replay to the image datarecorder having a low load factor. For example, the load factor of theimage data recorder 600-1 is 40% and is lowest. The command for a videoreplay is transferred to the image data recorder 600-1. In the roundrobin method often used or in the so-called method for sequentiallytransferring command for a video replay to the image data recorders600-1, 600-2, . . . 600-n for each occurrence of command for a videoreplay, the load factors of all the image data recorders can be almostuniform. Whether the operation of the load balancer 630 is done by atable lookup method or by the round robin method is decided by thedesign of the image system or the used state of command for a videoreplay.

The image data recorder which has received the transferred command for avideo replay, for example, the image data recorder 600-1, searches theID table 610 based on the ID inserted to the command for a video replay.When there is the corresponding record, the image data is read from thehard disk based on the position information Am in the image data storagearea 620 written thereinto to transmit it to the client terminal 800.The received image data is decoded by the decoder 820 for suitablyperforming reduced image processing and is displayed as the scaled-downimage 854 on the retrieval result list display screen 852 on the display830.

The reproduction screen 853 on the display 830 will be described. Thereproduction screen 853 shows a screen reproducing image data (imagedata picked-up by the camera and recorded) recorded into the videoserver 600. For example, when the user clicks the scaled-down image854-12 of the retrieval result list display screen 852 by a mouse,command for a video replay is inputted to the load balancer 630 togetherwith the ID attached to the scaled-down image 854-12. As in the displayof the above-described scaled-down image 854, the command for a videoreplay is suitably transferred according to the load state of the imagedata recorders 600-1, 600-2, . . . 600-3 to reproduce necessary imagedata. The reproduction screen 853 shows the case of reproducing imagedata of Mar. 12, 2002, 16:30:00:08. In the image data, a still image ora moving image can be reproduced and its replay speed can be freelychanged. When the reproduction operation is continuously performed, theabove processing is repeated. The image data recorder to which commandfor a video replay is transferred by the load balancer 630 is suitablyswitched by the load state (shown in Table 1) of each of the image datarecorders for the occasion. As described above, an ID is shared. Thecorresponding image data can be readily outputted. The load balancingprocessing for the entire Cluster, that is, the video server 600, isrealized.

An ID inserted to the above image data will be described. When ensuringthat all IDs are not overlapped, any ID may be used. In consideration ofthe transmission capacity and the recorder capacity, the amount of datais desirably as small as possible. In view of the reproductionprocessing, when an ID inserted (or attached) is ordered and successive,the absence of missing can be easily decided so that it is desirable asan ID. For example, suppose that the client 800 requires image data by apredetermined ID. When ensuring that IDs are not overlapped and aninserted ID is quite at random, the image data recorder 600-1 which hasreceived command for a video replay cannot decide whether image datacorresponding to the required ID is within the image data recorder 600-1unless all IDs of the ID table 610 are checked. As described above, whenthe ID table 610 cannot be on the semiconductor memory because the harddisk capacity of the image data recorder 600-1 is very large and someIDs are arranged on the hard disk, the search time of the ID table is100 times longer than the semiconductor memory not including the table.The response in the case that there is no image data with a required IDis significantly lowered. When an ID inserted is ordered and successive,the header and last IDs of the ID table 610 are checked and whether arequired ID is within its range is checked, thereby easily deciding thepresence or absence of image data. Generation of an ordered andsuccessive ID is convenient for image data retrieval.

As an embodiment of an ID satisfying the above condition, a method foremploying a natural number will be described. The ID generator 900incorporates a counter (or an ID generation unit) for generating an ID.The counter is initialized to 1 at product shipping. The system isinstalled and operated, and each time ID insertion is required to the IDgenerator 900, the internal counter is incremented by 1 to output thecorresponding ID from the output unit, not shown. In this case,initialization of the counter must not be permitted during that at all.The ID is a finite bit number. When the counter is returned to 0 again,ID overlapping may occur. For example, in the case of a TV, when IDs aregenerated to all frame images for keeping on ID increment by 30 framesper second, a natural number of about 40 bits corresponds to frameimages for about 1159 years. No practical problem in ID insertionoccurs.

In combination of the ID with the MAC address (48 bits) as an inherentID well-known as network equipment, an only ID can be allocated at frameimage level. In this case, about 88 bits is necessary. The bit lengthreleased for each company shipping a network card in the MAC address is24 bits (=16,777,216). The use of this can generate an ID with a shorterbit length.

The system application range is limited. When 24 bits is allocated as aproduct number allocated to the network image transmission unit 200 orthe network camera 500 and 40 bits is allocated to the counter in the IDgenerator 900, a practically sufficient ID can be used by about 64 bits.The ID of such counter output can easily be an ID combined with a cameranumber or a sensor number.

In the surveillance system, the image picking up time of the camera isoften used as the ID of image data. A time code with the time as an IDtends to be considered to substantially satisfy the ID condition. Whenimage data from a plurality of cameras are supplied to the network 100,time shift occurs between the plurality of cameras. Unless this issolved, the time code cannot be used as an ID. A clock used for generalIT (Information Technology) equipment counts time using the oscillationof a crystal oscillator. Since each crystal oscillator has an error,time is shifted little by little. To cope with such problem, the network100 uses NTP (Network Time Protocol) to adjust the time of the equipmenton the network 100. The operation sets the clock forward or backward inthe IT equipment whose time is shifted.

The target image system handles image data. The minimum unit of anallowed time shift is 16 ms corresponding to one field (half of oneframe) of image data. In a general personal computer, a shift to thisdegree occurs every three to six days. It is difficult to ignore thetime shift. In general, the network 100 itself is a medium allowingdelay, which does not ensure that time correction of the NTP iscomplete. Consequently, when a time code is generated as an ID to imagedata, ID overlapping or missing occurs.

There is a currently practical electric wave clock as a very preciseclock. This corrects-time for each hour. The electric wave clock isprovided in the ID generator and a time code is used as an ID, therebysignificantly improving the above time problem. With correction for eachhour, an error during that occurs. It is desirable that a more preciseelectric wave clock appear. When such precise electric wave clock isrealized, a time code can be used as an ID. Currently, the unit pricesof the peripheral parts of the electric wave clock are still expensiveand are difficult to employ for an actual system. When these unit pricesare very inexpensive, a time code of the electric wave clock can besufficiently used as an ID.

FIG. 3 is a block diagram of assistance in explaining another embodimentof the present invention and shows a water level surveillance system ofa dam recording image data along with a sensor obtaining water levelinformation. An image transmission unit 200 in FIG. 3 has an encoder210, a network transmitter 220, and an ID generator 900. The numeral700-2 is a DB (Data Base) recorder (or data recorder) recording waterlevel information from a sensor 400. The numeral 710 denotes a datatable matching an ID inserted corresponding to image data with waterlevel information 720 corresponding to image data. A client terminal 800has a display 830, a decoder 820, a retrieval unit 840, and a retrievalresult processing unit 850. The same things as FIGS. 1 and 2 aredesignated by similar numerals. This embodiment shows the case that theID generator 900 is incorporated into the image transmission unit 200.Multicast (a multi-destination transmission method to specified sides)is performed to an image data recorder 600-1 and the DB recorder 700-2for saving a network transmission bandwidth.

The operation of the water level surveillance system shown in FIG. 3will be described. The outputs from a camera 300 and the sensor 400 areinputted to the image transmission unit 200. The analog image dataoutput from the camera 300 is A/D converted by the encoder 210, iscompressed and encoded to JPEG (Joint Photographic Experts Group) orMPEG by the network transmitter 220, and becomes IP data together withsensor information (water level information) from the water level sensor400. The above ID is generated in an image reference unit (for example,in one frame) by the ID generator 900 to be outputted to the network100.

The image data and the sensor information (water level information) towhich the IDs are inserted (or attached) are transmitted to the imagedata recorder 600-1 and the DB recorder 700-2. In the image datarecorder 600-1, as explained in FIG. 2, the image data is recorded intoan image data storage area 620 on the hard disk in the image datarecorder 600-1, and recording position information on the decided areais matched with the inserted ID for recording into an ID table 610.

The relation between the image data storage area 620 and the ID table610 is the same as that shown in FIG. 7. When image data is inputted, itis stored in an area 621-1 on the image data storage area 620. An ID (inthis case, #101) generated to the image data and position information A1on the area 621-1 are stored in a record 611-1 on the ID table. When newimage data is inputted, position information A2 of an area 621-2 and anID (#102) inserted to the image data are stored in a record 611-2.Thereafter, each time new image data is inputted, position informationAm (m=1, 2, . . . ) of the area 620 and IDs (#M) (M=101, 102, . . . )are sequentially stored in records 611-m. Each time image data istransmitted, the above processing is repeated. When there is no space inthe image data storage area 620, old data are sequentially overwrittenby the latest data. The ID table 610 adds new data according to this.When there is a record whose referred side is updated by overwriting ofthe image data storage area 620, it is erased. The image data is alsotransmitted to the DB recorder 700-2, but is not used in the DB recorder700-2 and is discarded.

When the sensor information (water level information) is inputted in theDB recorder 700-2, it is added as a new record to the table 710 managingsensor information. At this time, the ID attached thereto is also storedtogether in the same record. For example, when image data stored in thearea 621-1 on the image data storage area 620 is transmitted from theimage transmission unit 200, sensor information (in this case, indicatedby “water level of 505 cm”) and ID information (in this case, indicatedby “101”) are stored in a first record 711-1 of the table 710 managingsensor information. When new sensor information is received, sensorinformation (in this case, indicated by “water level of 254 cm”) and IDinformation (in this case, 102) are stored in a record 711-2. Each timenew sensor information is received, sensor information (in this case,“water level height” information) and ID information are sequentiallystored in a record 711. Each time data is transmitted, the aboveprocessing is repeated. When there is no space in the table 710 managingsensor information, old data are sequentially discarded and areoverwritten by the latest data to be updated. To the DB recorder 700-2,image data is multicast from the image transmission unit 200, which isnot recorded and is discarded. In this embodiment, the DB recorder 700-2is described. For example, the image data recorder 600-2 shown in FIG. 2serves as it. In particular, the DB recorder need not be discriminatedfrom the image data recorder at all.

A method for retrieving information recorded into the image datarecorder 600-1 and the DB recorder 700-2 will be described. When a DBquery from the user is given to the client terminal 800, the retrievalunit 840 transmits a command statement describing the query condition,for example, an SQL statement, to the DB recorder 700-2. For example,information query in which the DB query from the user is water level of300 cm or above is instructed. The DB recorder 700-2 which has receivedthe DB query searches the table 710 managing sensor information,collects records corresponding to the given condition, and sends backthem as query result to the client 800. In the above case, water levelinformation “505 cm” stored in the record 711-1 corresponding toinformation of a water level of 300 cm or above, an ID “101” attachedthereto, water level information “325 cm”, and an ID “104” attachedthereto are transmitted to the retrieval result processing unit 850.

The retrieval result processing unit 850 isolates and extracts the waterlevel information and the ID from the received query result, processessensor information display data so as to display it to the user in asuitable form, and displays it on the display 830. In relation to thewater level information, when dam image data is necessary, IDs attachedto the water level information, or “101” and “104” in the above case,are transferred to the image data recorder 600-1 together with an imagequery. The image data recorder 600-1 searches the ID table 610 based onthe IDs “101” and “104” with the image query. After the search, whenthere is the corresponding record, image data is read from the imagedata storage area 620 based on position information written thereintoand is transmitted to the client 800. In the above case, image data areread based on the position information A1 and A4 corresponding to theIDs “101” and “104” from the corresponding areas 621-1 and 621-4 of theimage data storage area 620 and are transferred to the decoder 820.

The decoder 820 decodes the transmitted image data, processes it so asto display it on the display 830, and displays it on the display 830together with the water level information from the retrieval resultprocessing unit 850. As shown in FIG. 8, the display form can bedisplayed as a reduced image 854 on a retrieval result list displayscreen 852 by a known method, or can be displayed as a still image ormoving image of the picked-up image data on a reproduction screen 853.In this embodiment, the image data recorder 600-1 and the DB recorder700-2 are described as independent node servers. As described in FIG. 2,these servers are subject to clustering and can be operated in the loadbalanceable state.

Another embodiments of the present invention will be described usingFIGS. 4, 5 and 6. FIGS. 4 to 6 are schematic configuration diagrams ofthe embodiments of the present invention classified depending on in whatplace of the image recording and reproducing system an ID generator 900is located.

In FIG. 4, an ID generator 900 is provided in the same location asdescribed in FIGS. 2 and 3. By way of example, a plurality of videoservers 600 are connected to a network 100, the amount of data inputtedto a data recorder 700 is large so that it is unsuitable to transmitimage data, and it is difficult to match the data recorder 700 with amulticast protocol.

A network camera 500 transmits image data to a network video distributor1000 together with sensor information outputted from a sensor 400. Thenetwork video distributor 1000 is a unit isolating the sensorinformation superimposed on the image data from the image data anddistributing and transmitting them to the video servers 600 and the datarecorder 700. The network video distributor 1000 incorporates the IDgenerator 900 and inserts (or attaches) an ID to each of the isolatedimage data and sensor information. When a plurality of video servers 600exist, the image data and an ID attached thereto are multicast, asdescribed in FIG. 3. The sensor information is transmitted to the datarecorder 700 by an exclusive protocol. In FIG. 4, the network camera 500is physically separated from the network video distributor 1000. Thesemay be integrated. A unit combining a distribution function with an IDgeneration function is provided to make the network configuration in animage system flexible and to optimize the amount of informationtransmitted on a transmission line.

FIG. 5 is a schematic configuration diagram showing a systemconfiguration combined with an image recognition sensor. An imagerecognition sensor 1100 is connected to a network camera 500 and itsoutput is stored in a data recorder 700. Image data is stored in aplurality of video servers 600. In the image recognition sensor 1100 ofan intrusion surveillance system, when an intruder intrudes into aspecified location, the image recognition sensor 1100 senses it by imageprocessing and outputs information on the presence of an intruder.

In FIG. 5, the network camera 500 is integrated with an ID generator900. Without being limited to the network camera 500, it is integratedwith a camera 300. Image data picked-up by the network camera 500 and anID obtained by the incorporated ID generator 900 are both multicast tothe plurality of video servers 600 and the data recorder 700. The videoservers 600 record image data received by the same operation asdescribed in FIG. 3. The image recognition sensor 1100 which hasreceived the image data and the ID performs recognition processing fromthe inputted image data and outputs its recognition result as sensorinformation. At this time, the ID attached to the image data is attachedas-is to the sensor information and transmitted so as to maintain therelation of data between the video servers 600 and the data recorder700. The mutual processing of image recognition and a database which hasbeen realized only in an independent host can be realized by a networkenvironment very easily.

For example, the case of building a moving object detection system bythe system configuration of FIG. 5 will be considered. The imagerecognition sensor 1100 is equipped with a moving object detectionfunction. This can be realized by a prior art image processingtechnique. The brightness levels of the current image data and theprevious image data picked-up by the network camera 500 are compared todetect its change, thereby detecting a moving object. The imagerecognition sensor 1100 discriminates the moving object from theinputted image data and stores numerical values (What the moving body isexperimentally defined, previously.) indicating the position, size andcertainty of the moving object as sensor information in the datarecorder 700 by the above image processing.

A client terminal 800, omitted in FIG. 5, obtains image data from the IDincluded in the query result from the data recorder 700 and extracts thearea of the moving object using specified position and size informationto easily build a system creating a moving object list. When the systemis equipped with a human face extraction function, another set of theimage recognition sensor 1100 equipped with a face extraction functionand the data recorder 700 is prepared. The client terminal 800 searchesthe moving object database and the face image database. It can be easilyextended to a system retrieving image data with a moving object and itsface.

In a prior art independent host system, when adding the above face imagerecognition function, the CPU and memory of the host computer arelimited. It is difficult to easily extend the system.

FIG. 6 is a diagram of assistance in explaining the schematicconfiguration of an image system of further embodiment of the presentinvention. The system is a system realizing, by a minimum amount ofcommunication, image/sensor synchronism when a network camera 500 and asensor 400 exist to be juxtaposed. That is, it is a system generating ashared ID from an ID generator 900 to the network camera 500 and thesensor 400.

In FIG. 6, in order that the network camera 500 and the sensor 400 havesynchronism without mutual communication, the ID generator 900 isarranged in the uppermost stream and the ID is multicast from the IDgenerator 900. At this time, an ID interval is the same as the framerate of image data picked-up by the network camera 500. The networkcamera 500 which has received an ID transmits the ID and image data to avideo server 600 for recording them. The sensor 400 which has receivedan ID attaches the same ID as the ID attached to the image data tosensor information from the sensor 400 and transmits it to a datarecorder 700. Building such image system can easily generate a shared IDto image data and sensor information. An image system which can easilyrecord data, retrieve data and be extended can be realized.

With the principle of the present invention, a system seamlesslyswitching the network camera 500 and the video server 600 can berealized. When live image data is required by a client terminal 800, theclient terminal 800 is connected via a network 100 to the network camera500 to directly display image data picked-up by No. 1 camera of thenetwork camera 500 on the display of the client terminal 800. The clientterminal 800 may desire to fetch the past image data picked-up by theNo. 1 camera of the network camera 500 from the video server 600 to viewthe recorded image data. In this case, the processing must be moved insuch a manner that the connection must be instantly switched to thevideo server 600 to view the reproduced image data. In the system of thepresent invention, a shared ID is generated. The recorded image data ofthe No. 1 camera of the network camera 500 can be easily fetched fromthe video server 600 with reference to the ID.

To realize such system by the prior art technique, the client terminal800 must be constantly connected to the video server 600. Consequently,the picked-up image data from the network camera 500 is displayed formost of the time. System resources of the video server 600 must beconstantly allocated to the client terminal 800 whose access frequencyfor requiring the recorded image data from the video server 600 is low.The system resources are wasteful. In the present invention, only whenthe video server 600 is accessed, the video server 600 is connected. Theutilization efficiency of the video server 600 is increased. The videoserver 600 need not be operated constantly in synchronization. Thedesign is simplified. The system configuration can be simpler. Thesystem cost can be significantly reduced.

The present invention is described above in detail. The presentinvention is not limited to the image system, the recording andreproducing system for image data, and the recording and reproducingmethod therefor, which are described here. It can be widely applied toother image systems, recording and reproducing systems for image data,and recording and reproducing methods therefor.

The present invention has an effect of providing a recording andreproducing system for image data and a recording and reproducing methodtherefor capable of realizing duplicating or clustering of functions ofrecording, browsing and reproducing image data photographed by a camerain an image system and offering high reliability. That is, it canrealize a recording and reproducing system for image data and arecording and reproducing method therefor capable of equivalentlyaccessing a plurality of video servers. In addition, the presentinvention can provide a recording and reproducing system for image dataand a recording and reproducing method therefor capable of seamlesslyswitching live image data from a camera and recorded image data of avideo server while minimizing system resources such as a CPU, memory andbus bandwidth.

An ID inserted (or attached) to recorded image data and an ID insertedto sensor information are shared to facilitate the conjunctionprocessing of image data and sensor information, thereby minimizingsystem resources.

For duplicating or clustering, it is possible to provide a recording andreproducing system for image data and a recording and reproducing methodtherefor capable of correctly using recording database-related linkinformation when image data is distributively recorded into a pluralityof video servers.

An image system of the present invention has an effect of realizing arecording and reproducing system for image data and a recording andreproducing method therefor capable of facilitating video serveraddition or new function extension such as increasing of various piecesof sensor information of recognition and detection.

It is possible to provide a recording and reproducing system for imagedata and a recording and reproducing method therefor capable of usingrecording database-related link information without being affected bytime variation. Link information (ID) generated is ordered andsuccessive. It is possible to provide a recording and reproducing systemfor image data and a recording and reproducing method therefor capableof discriminating fast whether a predetermined part of recorded imagedata is recorded or not, without checking the entire recorded data.

Image data and link information (ID) are integrally generated. It ispossible to provide a recording and reproducing system for image dataand a recording and reproducing method therefor, which are high inreliability and inexpensive in cost.

It will be appreciated while particular embodiments of the inventionhave been shown and described, modifications may be made. It is intendedin the claims to cover all modifications which come within the truespirit and scope of the invention.

1. An image data recording and reproducing system comprising: an imagedata input unit for inputting an image data; an ID generator coupledwith said image data input unit, for generating an ID inserted into eachof video frames of said image data from said image data input unit,wherein said ID is inserted to distinguish each of said video frames,and each ID inserted into each of said video frames, and each IDinserted into each of said video frames, and each ID inserted into eachof said video frames is different from each other; a transmissionnetwork coupled with said image data input unit, for transmitting eachof said video frames with said each ID; a plurality of image datarecorders coupled with said transmission network, to each of which eachof said video frames with said ID is applied, wherein each of said videoframes with said each ID is commonly recorded into the plurality of saidimage data recorders, respectively, and wherein each of said pluralityof image data recorders has an ID table and an image data storage area,each of said video frames is recorded in said image data storage area,and position information of each of said video frames recorded in saidimage data storage area is stored in relation to said ID in said IDtable; and a plurality of terminal units coupled to said transmissionnetwork, each of which output each of said video frames recorded intosaid plurality of image data recorders, wherein when said ID is input inone of said terminal units, one of a plurality of said image datarecorders is selected and the video frame relating to said ID is outputfrom the selected one of said plurality of image data recorders to saidterminal unit in response to said ID, whereby one of said terminal unitsis able to output said video frame relating to said Id on the basis ofthe ID in said ID table from any one of said plurality of said imagedata recorders whenever a command relating to said ID for a video replayis input into one of said terminal units.
 2. An image data recording andreproducing system according to claim 1, further comprising: a loadbalancer which detects a state of load of said image data recorders,said load balancer having a load factor table, coupled with saidplurality of image data recorders, for selecting one of said pluralityof said image data recorders on the bases of the load factor of saidload factor table to output one of said video frames, wherein said loadfactor relates to a number of command applied from said terminal units.3. An image data recording and reproducing system according to claim 1,further comprising: a sensor information input unit, and a datarecording unit for recording sensor information from said sensorinformation input unit, wherein the ID from said ID generator isinserted to sensor information from said sensor information input unitso that said sensor information with the ID inserted is recorded intosaid data recording unit.
 4. An image data recording and reproducingsystem according to claim 1, wherein based on a retrieval from each ofsaid input terminal units, said ID table of said image data recorders isreferred to reproduce predetermined one of said video framescorresponding to an ID of said ID table.
 5. An image data recording andreproducing system according to claim 1, wherein an integer of 40 bitsor more is used for the ID outputted from said ID generator.
 6. An imagedata recording and reproducing system according to claim 2, wherein aninteger of 40 bits or more is used for the ID output from said IDgenerator.
 7. An image data recording and reproducing system accordingto claim 3, wherein an integer of 40 bits or more is used for the IDoutputted from said ID generator.
 8. An image data recording andreproducing system according to claim 1, wherein said image data inputunit includes an image data generation unit, wherein said ID generatorgenerating the ID inserted to each of said video frames from said imagedata generation unit is integrated with said image data generation unit.9. An image data recording and reproducing system according to claim 3,wherein the ID inserted to each of said video frames from said imagedata input unit and the ID inserted to sensor information from saidsensor information input unit are an ID from a shared ID generator. 10.An image recording and reproducing system according to claim 1, wherein,said ID generator comprises: an ID generation unit for generating asuccession of IDs of a successive integers.
 11. An image data recordingand reproducing system according to claim 8, wherein said image datageneration unit is a camera, and said ID generator is integrated withsaid camera.
 12. A method for recording and reproducing image data, saidmethod comprising the steps of: generating a plurality of video framesof said image data; generating an ID inserted into each of said videoframes; adding said ID to each of said video frames, wherein said ID isadded to distinguish each of said video frames and each ID added to eachof said video frames is different from each other; transmitting each ofsaid video frames with said each ID; recording each of said video frameswith said each ID added into a plurality of image data recorders,wherein each of said video frames with said each ID is commonly recordedinto the plurality of said image data recorders, respectively, andwherein each of said plurality of image data recorders has an ID tableand an image data storage area, each of said video frames is recorded insaid image data storage area, and position information of each of saidvideo frames recorded in said image data storage area is stored inrelation to said ID in said ID table; selecting one of a plurality ofsaid image data recorders when said ID is applied to one of a pluralityof terminal units; and reproducing the video frame relating to said IDfrom the selected one of said plurality of image data recorders inresponse to said ID, whereby one of said terminal units is able tooutput said video frame relating to said ID on the basis of the ID insaid ID table from any one of said plurality of said image datarecorders whenever a command relating to said ID for a video replay isinput into one of said terminal units.
 13. A recording and reproducingmethod according to claim 12, further comprising the steps of: in thecase of said command for a video replay, detecting a state of load ofeach of said plurality of image data recorders, and transferring saidcommand for a video replay to any one of said plurality of image datarecorders based on said state of load of said plurality of image datarecorders, wherein said state of load relates to a number of saidcommand.
 14. A recording and reproducing method according to claim 12,further comprising the steps of: obtaining sensor information related toeach of said video frames, inserting said ID to said detected sensorinformation, and recording sensor information with said ID into saidimage data recorders.